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Abstract: 


The  protection  of  perimeters  in  national, 
agricultural,  airport,  prison,  and  military  sites, 
and  residential  areas  against  dangerous 
approaching  human  and  vehicles  when  using 
human  agents  to  provide  security  is  expensive  or 
unsafe.  Because  of  this,  acoustic/vibration 
signature  identification  of  approaching  human 
and  vehicles  threats  has  attracted  increased 
attention.  This  paper  addresses  the  development 
and  deployment  of  three  types  of  acoustic  and 
vibration  based  smart  sensors  to  identify  and 
report  sequential  approaching  threats  prior  to 
the  intrusion.  More  specifically,  we  have 
developed:  a)  acoustic  based  long  range  sensor 
with  which  vehicles'  engine  sound  and  type  can 
be  identified,  b)  vibration  based  seismic  analyzer 
which  discriminates  between  human  footsteps 
and  other  seismic  events  such  as  those  caused  by 
animals,  and  c)  fence  breaching  vibration  sensor 
which  can  detect  intentional  disturbances  on  the 
fence  and  discriminate  between  climb,  kick, 
rattle,  and  lean.  All  of  these  sensors  were 
designed  with  several  issues  in  mind,  namely, 
optimized  low  power  usage,  a  low  number  of 
false  positives,  small  size,  secure  radio 
communication,  and  milspec.  The  developed 
vibration  based  system  was  installed  in  an 
airport  with  unprotected  shore  lines  in  the 
vicinity  of  taxi-  and  run-ways.  The  system 
reported  an  average  of  less  than  two  false 
positives  per  week  and  zero  false  negative  for  the 
duration  of  forty-five  days.  Six  fence  sensors 
were  installed  on  the  terminal  area  and  end-of- 
runway  chain-link  fences  where  there  was 
possibility  of  intentional  fence  climbing.  The 
fence  sensors  reported  no  false  positives  for  the 
duration  of  forty-five  days  which  included 
several  days  of  seasonal  storms. 


INTRODUCTION 


The  increasing  emphasis  on  perimeter 
protection  of  national  assets,  both  at  home 
and  abroad,  has  spurred  the  development  of 
technologies  that  can  detect  potential  threats, 
such  as  humans  or  approaching  vehicles.  One 
area  of  interest  is  the  combination  of  several 
different  sensors,  each  with  their  own  distinct 
modalities  and  detection  ranges,  to  create  a 
versatile  and  robust  system.  We  propose  a 
system  that  would  be  based  on  three  security 
sensors  which  have  been  demonstrated  to 
provide  detection  and  classification  of 
security  threats.  These  sensor  types  are 
acoustic,  seismic,  and  vibration  sensors. 

The  "Smart  Fence"  system  based  on  these 
sensors  would  be  particularly  suited  for  the 
identification  and  reporting  of  sequential 
approaching  events,  such  as  an  approaching 
vehicle  with  passengers  desiring  to  breach  a 
fenced  facility.  One  benefit  is  that  the 
likelihood  of  detection  of  intruders  is 
increased  because  the  sensors  are  designed 
for  detection  of  different  characteristics  of  an 
intrusion  attempt.  For  example,  if  one  sensor 
misses  detection  due  to  some  confounding 
event,  other  sensors  may  not  be  susceptible 
to  such  an  event,  providing  another  chance 
for  detection.  Another  benefit  is  that  the 
sensors  allow  the  system  to  deliver  threat- 
level  information  about  incoming  threats.  For 
example,  vehicles  detected  at  a  long  distance 
may  have  a  lower  threat-level  than  intruders 
approaching  a  secure  perimeter,  which  in 
turn  may  have  a  lower  threat  level  than  the 
intruders  attempting  to  scale  a  fence 
surrounding  the  perimeter. 

The  acoustic  signatures  of  interest 
generated  by  approaching  human  and 
running  vehicles  are  complicated.  In  the  case 
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of  running  vehicles,  the  acoustic  patterns  are 
affected  by  multiple  factors,  such  as  gearing, 
number  of  cylinders,  muffler  choice,  state  of 
maintenance,  running  speed,  distance  from 
the  microphone,  tires,  the  road  on  which  the 
vehicle  travels,  and  other  uncontrolled 
environmental  interference  emitted  by  wind, 
bird  chirp,  and  human  voice.  We  propose 
using  a  neurobiology-motivated  algorithm  to 
detect  approaching  vehicles  and  to  identify 
the  type  of  vehicles.  With  the  exact  acoustic 
signature  being  unknown.  Nonlinear 
Hebbian  learning  (NHL),  a  basic  and 
appealing  neural  learning  function  found  in 
human  brain,  is  employed  for  unsupervised 
learning.  The  developed  system  is  capable  of 
discriminating  between  three  types  of 
vehicles,  namely  light  track,  heavy  track,  and 
motor  cycle. 

The  goal  of  a  seismic-based  human  threat 
detector  is  to  detect  approaching  humans  and 
discriminate  between  series  of  events  caused 
by  animals  or  passenger  vehicles  vs. 
background  and  a  single  vibration  event,  e.g. 
the  falling  of  a  tree  limb.  A  geophone-based 
seismometer  has  been  employed  which  is  an 
inexpensive  sensor  that  provides  easy  and 
instant  deployment  as  well  as  long  range 
detection  capability.  Gaussian  mixture 
models  were  formed  to  model  statistical 
properties  of  the  temporal  gait  and  frequency 
features  extracted  from  the  seismic  signals. 
The  system  was  set  up  to  discriminate 
between  human  footsteps,  vehicles, 
background  (including  disconnected 
incidents,  animals’  footsteps). 

For  the  purpose  of  fence  breach  detection 
and  classification,  a  3-axis  accelerometer  has 
been  utilized.  The  developed  algorithm  which 
is  based  on  a  nonhomogeneous  Markov 
model  is  capable  of  recognizing  the  type  of 
the  breaches;  whether  the  breach  is  due  to 
the  rattling  caused  by  strong  wind  or  a  person 
climbing  on  the  fence.  The  proposed 
algorithm  and  system  has  been  tested  on 
different  fences  and  has  demonstrated  robust 
recognition  for  discriminating  between  climb, 
kick,  rattle,  and  background. 


Method 

Seismic  Event  Recognizer:  seismic 
discrimination  between  human 
footsteps,  vehicle,  and  out-of-class 
signals. 

The  signal  measured  from  a  geophone  has  a 
o.lHz~looHz  frequency  range  due  to  the 
resonant  characteristics  of  the  sensors. 
Although  the  frequency  response  of  the 
seismic  sensor  is  in  a  narrow  frequency  band, 
spectral  analysis  can  be  used  for 
discriminating  between  seismic  events 
caused  by  human  footsteps  (or  four-legged 
animals)  and  vehicles.  However,  due  to  the 
very  similar  walking  mechanism  of  humans 
and  animals,  the  generated  rhythmic 
temporal  seismic  patterns  of  humans  and 
animals  are  very  similar.  This  renders  the 
discrimination  between  a  human's  and  an 
animal's  footstep  using  frequency  analysis  a 
failure  using  spectral  analysis  alone. 
Therefore  we  propose  using  gait  pattern 
analysis  of  seismic  events  to  discriminate 
between  human  and  animal  footsteps.  ^ 
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Figure  1 :  Feature  extraction  algorithm  and  examples  of 
the  output  of  each  block. 


Homeland  Security  Affairs,  Supplement  3  (March  2011)  WWW.HSAJ.ORG 


Dibazer,  etal,  Intelligent  Acoustic 


3 


The  flow  diagram  of  the  method  is 
depicted  in  the  Figure  i.  After  applying  a 
three-second  sliding  window  -  with  two 
seconds  overlap  -  on  the  incoming  signal,  the 
signal  is  passed  through  a  band-pass  filter  to 
enhance  the  Signal  to  Noise  Ratio  (SNR). 
Then  a  Hilbert  transform  and  low  pass 
filtering  (smoothing  process)  is  applied  to 
extract  the  envelope  of  the  signal.  In  the  next 
step,  we  utilize  this  signal  to  extract  the  mean 
temporal  pattern  of  the  gait  by  averaging  over 
each  gait  periods. 

Accordingly  it  is  necessary  to  estimate  the 
gait  period  within  the  three-second  window 
and  partition  the  three  seconds  signal  based 
on  gait  period.  This  can  be  achieved  in  the 
following  steps: 

Gait  period  is  estimated  by  using  the  auto¬ 
correlation  function.  Because  of  the 
periodicity  in  the  signal,  the  auto-correlation 
signal  has  a  local  maximum  at  the  time  of  gait 
period.  In  general,  finding  local  maxima  is  a 
challenge;  however,  due  to  the  resonant 
characteristics  of  the  seismic  sensors  and  the 
periodicity  from  walking  mechanism,  there  is 
a  detectable  peak  in  the  auto-correlation 
function.  It  also  worth  mentioning  that  the 
gait  period  (or  cadence  frequency)  will  later 
be  employed  as  one  of  the  features. 

Using  the  estimated  gait  period  in  A,  the 
three  second  window  is  equally  divided  into  k 
number  smaller  windows  each  having  gait 
period  length. 

The  partitioned  signals  from  B  are 
averaged. 

In  order  to  make  a  shift-invariant 
temporal  gait  pattern  representation,  the 
averaged  gait  pattern  from  C  is  circular- 
shifted  so  that  the  local  maximum  of  the 
pattern  is  on  the  first  sample.  The 
partitioning  of  the  three  second  signal  into  k 
frames  will  have  some  remainder  which  is 
considered  in  the  circular  shift  of  the  next 
consecutive  frame. 

Lastly,  twelve  triangular  weighting 
functions  are  applied  to  the  temporal  pattern 
acquired  from  the  steps  explained  above  so 
that  the  gait  temporal  pattern  can  be 
represented  by  twelve  features. 

For  modeling  statistical  variation  of  the 
extracted  features,  Gaussian  Mixture  Models 
(GMM)  were  utilized.  GMM  is  one  of  the 
most  well-known  and  useful  classifiers, 
having  been  widely  used  in  many 


applications.  For  a  multimodal  random 
variable,  whose  values  are  generated  by  one 
of  several  independent  sources,  a  finite 
mixture  model  can  be  used  to  approximate 
the  true  probability  density  function. 
Moreover,  GMM  is  a  good  candidate  as  a 
classifier  when  there  exists  no  prior 
knowledge  of  a  probability  density  function. 
Therefore,  estimating  the  distribution  with 
GMM  not  only  provides  a  chance  to  have  a 
general  model  but  also  helps  to  understand 
the  phenomena  for  a  better  use  of  the 
information  of  the  distribution. 

A  non-singular  multivariate  normal 
distribution  of  a  D  dimensional  random 
variable  X^x  can  be  defined  as: 

X~  N(x,.  n,  , SJ  = 

^)^E-'(x-p)]  (1) 

where  is  the  mean  vector  and  X  the 
covariance  matrix  of  the  normally  distributed 
random  variable  X. 

The  GMM  can  be  defined  as  a  weighted 
sum  of  Gaussians  function: 

p(x,  e)  =  Ec=l  «c  ^ (Xc,  lie,  Sc)  (2) 

where  ac  is  the  weight  of  mixture  and  6  is 
defined  as  following. 

9  =  {ai,  Xi, ... ,  ac,  He,  Sc}  (S) 

To  estimate  or  train  the  model  parameter 
6,  the  Figueiredo-Jain  (FJ)  algorithm  was 
used,  3  which  automatically  chooses  the 
optimum  number  of  mixtures  during  the 
training.  The  objective  function  of  this 
algorithm  utilizes  the  minimum  message 
length  criterion  for  finding  optimum  number 
of  mixtures  as  defined  in  the  equation  (4)  so 
that  it  can  select  the  best  model  directly  from 
data  rather  than  the  hierarchy  of  model-class. 

A(9.>r)=iZ..>oln®  +  ^lng)  + 

-  ln£(A',  0)  (4) 

where  N  is  the  number  of  training  points,  V  is 
the  number  of  free  parameters  specifying  a 
component,  and  Cnz  is  the  number  of 
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mixture  (ac>  0).  The  last  term,  ^),  is 

the  log-likelihood  of  the  training  data  given 
the  distribution  parameters  6. 

Nonhomogeneous  Markov  Model  for 
fence  intrusion  detection  and 
discrimination  between  fence 
climbing,  rattle,  kick,  and  background 

The  sensing  module  in  the  fence  sensor  is  a  3- 
axis  digital  accelerometer  (X,  Y  and  Z  axes) 
measuring  the  vibration  signal  acceleration 
range.  The  source  of  kick  and  lean  intrusions 
are  short-time  forces  imposed  on  the  fence. 
As  a  result,  the  event  starts  with  a  high-level 
acceleration  and  high-frequency  vibrations. 
As  time  progresses,  the  acceleration  decays 
exponentially  and  vibration  gets  a  harmonic 
structure.  In  climbing,  rattling,  and  scratch 
intrusions,  the  vibration  source  has  longer 
time  periods.  The  decay  pattern  and  the 
transition  from  high-frequency  to  harmonic 
vibration  are  less  prominent.  The  intrusion 
signal  analysis  demonstrates  the  rattling 
signal  to  be  more  or  less  periodic,  while  the 
climb  and  scratch  signals  are  chaotic. 

The  signal  time-domain  envelope  plus  its 
time-frequency  features  are  used  in  intrusion 
classification.  The  time-domain  envelope  is 
represented  by  Z-axis  energy  plus  relative 
energy  in  X  and  Y-axis  to  Z-axis.  The  time- 
frequency  features  are  extracted  by 
measuring  Z  and  X  axis  signal  energy  in  a  set 
of  filter-banks.  Features  are  extracted  in  256- 
point  (400  ms)  windows  of  signal  with  50 
percent  (200  ms)  overlap. 
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Figure  2:  Z  axis  signal  for  different  intrusion  classes. 


Intrusion  signals  recorded  by  sensors 
demonstrate  temporal  and  dynamic  patterns, 
implying  the  use  of  HMM  as  a  standard 
dynamic  classifier.  A  trivial  technique  for 
intrusion  classification  deals  with  defining 
and  applying  one  HMM  for  each  intrusion 
class.  This  method  is  practically  inadequate, 
because  the  sensor’s  computational  resource, 
a  16  MIPS  RICS  processor,  is  not  sufficient 
for  implementing  five  HMM  models  plus 
their  post-processing  task  within  the 
classifier’s  time  resolution  (200  ms,  the 
signal  update  period).  To  go  beyond  this 
limitation,  we  utilized  a  NHMM  model  for 
intrusion  localization  along  with  a  static 
classifier.  The  new  classifier  not  only  has 
lower  computational  complexity,  but  also 
presents  a  higher  classification  performance 
comparing  to  the  trivial  method.  The  chief 
reason  for  the  complexity  reduction  plus 
performance  improvement  is  the  precise 
intrusion  localization  provided  by  NHMM. 
The  sequence  analyzer,  a  dynamic  classifier, 
is  designed  as  a  3-state  HMM  with  a  time- 
dependent  transition  probability  matrix.  It 
identifies  the  most  probable  time-window  of 
the  signal  for  different  intrusions.  Then  the 
static  classifier,  a  Bayesian  classifier, 
examines  the  likelihood  of  different 
intrusions  to  find  the  most  probable  intrusion 
class.  Data  analysis  shows  that  the 
performance  of  NHMM  in  intrusion 
localization  surpasses  that  of  homogenous 
HMM.  Moreover,  the  NHMM  generates 
higher  likelihood  values  than  HMM  in  signal 
sequences,  with  an  equal  state  definition  for 
both  models.  The  dynamicity  of  transition 
probability  matrix  brings  more  flexibility  in 
the  model,  allowing  it  to  follow  unusual  and 
abrupt  behaviors  in  the  intrusion  signal. 

The  logarithm  of  Z-axis  energy  in 
successive  frames  is  passed  to  the  transition 
probability  matrix  of  the  sequence  analyzer. 
The  energy  plus  time-frequency  features 
determine  the  sequence  observation  of  the 
analyzer.  Non-intrusion  (called  Si  state), 
harmonic  (called  S2  state),  and  high- 
frequency  (called  S3  state)  periods  determine 
three  states  of  the  model.  In  the  next  part,  we 
introduce  the  dynamic  transition  probability 
and  its  training  procedure.  4 

Assume  o  =  {o,  i  =  i,2,...,N}  is  the 
observation  generated  by  a  HMM  model  and 
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its  hidden  states  are  C  =  {a,,  i  -  1,2,  N}  . 
The  joint  distribution  of  observation  is, 

PiOlC.e)  = 

p(ci)p(oi  |ci,  6)  nf=2  P(Oi Iq,  0)p(Ci  |Ci_i)  (l) 


The  emission  process  p(o,jc,-,  is  considered 
equal  for  the  sequence  and  the  transition 
probability  matrix  pfCjjCj-l^  is  a  function  of 

observation  index,  i.  is  the  feature  set 
representing  data  frame,  and  9  determines 
the  state  parameter.  The  transition  matrix 
dependency  to  sequence  index  means  that 
the  hidden  states  follow  a  NHMM  chain.  To 
estimate  parameters  of  the  local  transition 
probability  matrix,  the  matrix  entries  are 
defined  as  a  function  dependent  to  a  set  of 
observed  features. 

The  transition  probability  matrix  is 
modeled  by  a  function  of  energy  in  adjacent 
frames.  Based  on  the  state  definition,  the 
transition  probability  matrix.  A,  is  a  3-by-3 
matrix  with  the  following  definition  for  its 
pg*  entry. 


tpq  fcpq  *  ^m-l)  “t  ^pq  *  ^m-l)  “t  ^pq 


^12 

%3‘ 

^21 

^22 

^23 

-%1 

%2 

%3. 

(2) 


Cm  and  em-i  are  energy  logarithms  in  m  and 
m-i  observation  frames.  The  Apq  entiy,  the 
probability  of  transition  from  state  p  to  state 
q,  is  a  soft-max  function  of  tpq,  for  p  =  {1,  2, 
3}  and  q  =  {1,  2,  3}.  {Qp,p=i,2,3}  is  the  set  of 
parameters  defining  the  A  matrix,  in  which 

ul  u2  u3 

Qp  -  The  tpq  is  defined  as 

quadratic  function  of  energy  logarithm’s 
difference,  representing  a  quadratic 
approximation  of  relative  energy  (cm/  Cm-i)  in 
matrix  A.  In  fact,  the  energy  trend  in 
successive  frames  has  been  modeled  with  a 
second  order  polynomial  approximation. 

Intuitively  the  coefficient  carries  the 

homogenous  part  of  A  entries,  while  the^P<7 
7,2 

and  '^p<7  mutually  determine  the  energy 

7,2 

dependent  part  of  A.  The  '^PQ  magnifies 


energy  difference  in  the  model,  while  ^PQ 
intensifies  energy  trend.  For  example  for  033, 

we  expect^33  to  be  a  positive  number 

reasonably  bigger  than^33.  This  is  because  in 
successive  S3  states,  energy  amplitude  decays 
by  index. 

The  training  process  estimates  optimized 
values  for  Qp  and  O  with  maximum  likelihood 
(ML)  criteria.  Each  state  is  modeled  with  a 
multivariate  Gaussian  function;  Ms,s={i,2,3}. 
The  training  procedure,  the  Baum-Welch 
algorithm,  maximizes  likelihood  of  the 
observed  data  by  adjusting  Qp  and  Ms 
parameters.  By  using  the  logarithm  of 
equation  (2),  the  state  and  transition 
probability  equations  are  factorized  to 
additive  components,  making  training 
procedure  simple. 


max 

Qi.Q2>Q3>1^iM2M3 


log  P(0\C,  6) 


oc  maxQ^,Q2.Q3,Mi,M2,M3(l]iLi  logPCoilq) 
a 

Sf=i  logP(Oilq)  + 


^^^qi.q2.Q3  q-i) 


+ 


(3) 


In  (3),  p(c; I  c~i)  was  replaced  by  matrix  A 
entries  and  p(cj)  was  set  to  a  constant 
number.  The  optimum  values  for  6  =  {Mi,  M2, 
M3},  state  parameters,  are  mean  and 
covariance  matrix  of  corresponding  observed 
states.  To  find  Qp  parameters,  the  second 
term  of  equation  (3)  can  be  factorized. 


‘"3Xq^,<32.<33  5^r=iiog^ciCj_i(q.q-i)  = 

maxQ^,ci_i65i  Sf=2logAc;ci_i(ei.ei-i)  + 
maxQ^,Ci_ie52  log^Ciq.iCef,  q-i)  + 
maxg3,<.i_,653  Sf=2  log^CiCi_i(ei,  q-i)  (d) 


According  to  (4),  parameters  of  Qi,  Qg,  Q3  can 
be  optimized  separately.  By  expanding  each 
term  of  (4)  using  Apq  definition,  ML  estimate 
of  the  model  turns  to  be  an  unconstrained 
optimization  for  corresponding  Qp 
parameters.  The  optimum  parameters  of  Qp 
can  be  found  by  gradient  ascent  method. 

Training  procedure  starts  with  a  Viterbi 
algorithm,  5  extracting  the  most  probable 
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parameters  are  estimated  from  the  decoded 
states.  The  training  process  repeats  until  the 
likelihood  improvement  of  ohservation  drops 
to  a  certain  level  or  the  decoded  ohservation 
has  no  significant  state  changes.  In  the  model 
definition,  we  assigned  a  specific  state  (Si,  S2 
and  S3)  to  each  part  of  the  signal.  Training 
signals  are  roughly  labeled  with  their 
matching  states;  as  a  result  the  training 
initialization  step  starts  with  desired  state 
definitions  (a  semi-supervised  learning). 

After  training,  the  intrusion  signals  can  be 
decoded  with  the  Viterbi  algorithm  to  localize 
intrusions.  Figure  2  displays  the  extracted 
transition  in  a  test  signal.  The  recorded  signal 
contains  forty  seconds  of  vibration  in  a  slack 
fence.  It  contains  a  combination  of  intrusion 
attempts  starting  with  two  kicks  and 
following  by  two  leans,  climbing,  two  scratch 
periods.  In  Figure  2,  intrusion  events  have 
been  localized  precisely  (within  200  msec  of 
the  actual  event). 

The  localization  result  is  passed  to  a  state 
machine,  a  part  of  the  sequence  analyzer.  The 
state  machine  determines  different  feature 
sets  which  are  passed  to  the  different 
intrusion  classes  of  the  classifier.  The  static 
classifier  identifies  intrusions  by  averaging 
likelihood  values  of  two  Bayesian  classifiers. 
The  first  classifier’s  input  includes  time- 
frequency  features  of  the  signal  and  the 
second  classifier  processes  envelope  features. 
The  classifier  output  is  the  intrusion  class 
with  maximum  likelihood. 


Time 


Acoustic  signature  recognition  of 
approaching  vehicle  using  Nonlinear 
Hehbian  Learning 

The  proposed  approach  for  this  research  is 
noise-independent  acoustic  signature 
identification  and  recognition  using  spectro- 
temporal  dynamic  neural  representation  and 
Nonlinear  Hehbian  Learning  (NHL).^  As 
stated  in  the  introduction,  the  acoustic  sound 
of  interest  from  a  running  vehicle  is  affected 
by  multiple  factors,  such  as  gearing,  number 
of  cylinders,  muffler  choice,  state  of 
maintenance,  running  speed,  distance  from 
the  microphone,  tires,  and  the  road  on  which 
the  vehicle  travels.  As  the  sounds  of  interest 
from  various  types  of  vehicles  may  be  defined 
by  some  common  factors,  and  highly 
correlated  between  different  types,  vehicle 
type  identification  is  also  complicated 
because  of  the  presence  of  uncontrolled 
interference  emitted  by  surrounding 
background,  such  as  human  voice,  gunshots, 
and  wind. 

The  preprocessing  of  the  proposed  method 
mimics  functions  of  the  biological  ear  to 
correctly  recognize  signals  of  interest,  even  if 
they  are  noise-corrupted.  The  signals  of 
interest  are  acoustic  sounds  of  approaching 
vehicles  -  light  track,  heavy  track,  diesel 
truck,  and  motorcycle  -  which  are  often 
overlapped  with  environmental  noises.  In 
general  these  noises  are  highly  time-varying 
and  unknown  to  the  recognizer.  State-of-the- 
art  feature  extraction  algorithms,  such  as  Mel 
Frequency  Cepstral  Coefficients  (MFCC)  fail 
in  this  case,  because  MFCC  work  only  in 
short-term  spectral  analysis. 


Figure  3:  Top  is  a  test  signal  and  bottom  is  the  output 
of  sequence  analyzer 
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Figure  4:  Flow  diagram  of  the  proposed  vehicle's 
engine  sound  recognizer 


As  illustrated  in  Figure  5,  the  left  module 
represents  the  spectro-temporal  dynamic 
neural  structure  with  Q  frequency  hins  and  M 
temporal  frames.  Selecting  Q=30,  M=20,  the 
dimension  of  spectro-temporal  features  is 
600,  which  may  cause  very  complex 
computation  at  the  testing  stage  if  they  are 
used  as  patterns.  Besides,  high-dimensional 
features  may  he  less  useful  than  real 
representative  features  as  high-dimensional 
ones  may  he  easily  mixed  with  unrelated 
noises. 


weights 


Figure  5:  Nonlinear  Hebbian  illustration 

Linear  Hehhian  Learning  (LHL)  which  is 
equivalent  to  linear  principal  component 
analysis  can  he  used  for  dimensionality 
reduction;  hut  in  an  information-theoretic 


context,  the  second-order  moment  of  LHL  is 
inadequate  to  reduce  data  redundancy,  as 
generally  the  mutual  information  between 
important  components  of  data  involves 
statistics  of  all  orders.  Thus,  to  tackle  the 
curse  of  dimensionality,  a  NHL,  an 
enhancement  of  LHL,  is  proposed  to  be 
capable  of  efficiently  projecting  this  messy 
high-dimensional  representation  to  a  low¬ 
dimensional  subspace.  As  a  result,  the  NHL 
captures  important  features  while  removing 
unimportant  ones. 

NHL  iteratively  updates  neuron  outputs 
and  synaptic  weights  via  the  following  two 
steps.  Upon  convergence,  representative 
independent  features  (signatures)  are 
extracted  and  the  projecting  subspace  is 
spanned  by  synaptic  weight  vectors. 

Step  I:  Neuron  output  (signature) 
computation 

fl  =  Eq  =  i  'Zm=l  WqmlXqm  .  ^  G  [1.  -  .  L]  (S) 
Step  II:  Synaptic  weight  update 

=  r|g(yi)g'(yi)(Xqm  “  Si=l  Wq^iyi),  q  G 
[1 . Q],mG[l . M],1G[1 . L]  (6) 

where  Q  and  M  are  the  number  of  spectral 
bins  and  temporal  frames,  respectively.  L  is 
the  number  of  extracted  representative 
patterns.  Wqm\  represents  the  connecting 
spectro-temporal  synaptic  weight  from  the 
input  neuron  Xqm  to  the  output  neuron  yi . 

The  nonlinear  activation  function  g(.)  is 
the  critical  part  in  Nonlinear  Hebbian 
Learning.  If  it  is  Taylor  expanded,  it  can 
explore  all  order  statistics  of  input  signals 
and  satisfy  statistical  optimization  criterion. 
Moreover,  this  special  function  is  derived 
from  neurobiology  studies,  which  supports 
multiple  independent  signal  communications 
in  response  to  pre-synaptic  inputs. 

Results 

The  mathematical  models  in  each  technology 
were  trained  with  the  data  recorded  in  a 
controlled  environment.  The  recording 
samples  for  each  class  of  intrusion  were 
mutually  exclusive  from  the  other  classes.  All 
of  the  recording  samples  for  the  purpose  of 
this  study  were  collected  in  Joshua  Tree,  CA 
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for  which  more  details  can  be  found  in  the 
referred  papers. 

Two  types  of  test  were  planned.  The  first 
plan  was  to  evaluate  the  smart  fence  sensors 
in  the  Joshua  Tree  recording  site  where  the 
training  samples  were  collected.  The  second 
plan  was  to  install  the  sensor  in  a  completely 
realistic  condition  where  the  impact  of 
environmental  factors  such  as  storm, 
moisture,  wind,  heat,  and  etc  can  also  be 
measured.  The  later  provides  potential  to 
observe  the  behavior  of  the  sensors  for 
unexpected  incidents  -  such  as  those  training 
samples  not  available  during  the 
development  -  and  also  monitor  physical 
parameters  of  the  sensors  e.g.,  battery  life 
and  wireless  communication  issues. 

Test  Results  in  Controlled 
Environment 


Seismic  sensor 

For  testing  seismic  sensor’s  performance,  the 
sensor  was  placed  in  the  same  site  in  which 
the  training  samples  were  recorded.  During 
the  test,  the  average  of  posteriori 
probabilities  of  each  class  on  ten  consecutive 
window  frames  was  calculated  (it  has  been 
assumed  that  there  is  no  abrupt  changes 
within  the  intrusion  class).  We  found  that  the 
average  posteriori  is  a  powerful  technique  in 
this  application  which  enhances  the  low-SNR 
observations  results  and  reduces  false 
positives. 

A  horse  was  chosen  for  a  quadruped  class 
because  the  gait  can  be  easily  controlled  by  a 
rider  and  also  data  can  be  easily  acquired 
with  the  rider's  control.  The  seismic  sensor 
was  tested  with  different  types  of  the  horse 
gaits,  namely  walk  as  a  4-beat  gait,  trot  as  a 
2-beat  gait,  canter  as  a  3-beat  gait,  and  gallop 
as  a  fastest  4-beat  gait.  In  addition,  the 
seismic  sensor  was  also  evaluated  with  a 
single  person  walking/running  and  five 
people  walking  in  a  group  at  synchronized/ 
unsynchronized,  random  speed,  and  random 
phase.  Moreover,  multiple  vehicles  with  a 
different  number  of  cylinders  and  frame  size 
were  employed  to  evaluate  the  sensor.  The 
results  of  the  tests  are  summarized  in  the 
Table  1. 


0  0.2  04  0.6  0.8  1  1.2  14  1.6  1.8  2 


xio' 

Figure  6:  Bottom  graph  is  seismic  recognition  of 
background  (no  bar),  human  footstep  (red  bars),  dog’s 
footstep  (black  bar),  vehicle  (blue  bar),  and  horse  (cyan 
bar).  Top  is  the  spectrum  of  the  signal. 


Table  1:  Performance  of  seismic  recognizer  in  the 
Joshua  Tree,  CA  recording  site;  one  frame  is  400msec 


Background 

Vehicle 

Human 

Quadruped 

False  Recog. 

% 

0.1 

0.2 

3.8 

1.9 

Total  Frames 

19470 

61110 

26440 

30380 

Fence  Sensor 

One  fence  sensor  was  placed  on  a  variety  of 
fences  to  evaluate  the  performance.  Fence 
heights  ranged  from  4’  to  12’,  widths  from  6’ 
to  12’,  with  variable  tightness  and  looseness, 
variable  degrees  of  sagging,  with  and  without 
top  and  bottom  rails,  and  variable  diameters/ 
strength  of  holding  posts. 

Five  male  intruders  were  asked  to  disturb 
the  fence  -  one  at  a  time  -  and  generate 
desired  classes  of  intrusions.  The  intruders 
weighed  from  125  to  210  pounds  and 
demonstrated  various  patterns  of  climb  and 
other  intrusion  classes.  The  results  of  the  test 
have  been  summarized  in  the  Table  2. 


Figure  7:  Recognized  fence  events;  background  (red 
bars),  rattle  (green  bar).  Kick  (black  bars),  climb  (cyan) 
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Table  2:  Performance  of  the  fence  sensor  evaluated  on 
variety  of  fences;  duration  of  an  event  can  be  3  to  10 
seconds  depending  on  the  type  of  fence  and  class  of 
intrusion 


Test  Results:  Naval  Surface 
Warfare  Center’s  Airport 
Installation 


Background 

Rattle 

Climb 

Kick 

False  Recog. 

% 

0.2 

0.9 

1.8 

2.1 

Total  Events 

3238 

2210 

1045 

1051 

Vehicle’s  Engine  Sound  Recognizer 

The  proposed  NHL  approach  was  utilized  to 
(a)  recognize  vehicle’s  engine  sound  and  (b) 
detect  type  of  engine.  For  the  purpose  of  the 
test,  light/heavy  track  vehicles  fueled  with 
gasoline  were  employed  in  addition  to  an 
eight  cylinder  diesel  truck  and  25OCC 
motorcycle. 

The  summarized  report  of  vehicle 
identification  has  been  illustrated  in  the 
Table  3.  A  noise  source  of  human  speech, 
bird  chirp,  or  pink  noise  was  located  next  to 
the  recording  microphone  in  order  to 
generate  the  desired  signal  to  noise  ratio. 


Figure  8:  Top  is  the  microphone  output  of  a  passing 
vehicle,  and  bottom  Is  the  output  of  the  recognizer. 


Table  3:  Performance  of  vehicle  identification  in  the 
Joshua  Tree,  CA  recording  site. 


Gasoline 

Diesel 

SNR 

Light  Track  Heavy  Track  Diesel  Truck  Motorcycle 

10  db 

90% 

95% 

95%  100% 

5db 

80% 

90% 

95%  95% 

The  developed  smart  sensors  were  installed 
in  an  airport  where  some  of  its  land  is 
surrounded  by  water  and  there  is  no  fence 
line  to  prevent  intruders  who  park  their  boats 
along  the  shore  and  walk  to  the  runway. 
Several  footstep  sensors  and  one  vehicle 
engine  sound  recognizer  were  installed  in 
front  and  on  the  roof  of  the  boathouse 
respectively  (see  Figure  9).  Moreover,  several 
fence  and  footstep  sensors  were  installed  at 
the  south  end  of  runway  where,  due  to  its 
proximity  to  a  freeway,  the  sensors  would 
have  the  highest  chance  of  detecting 
intrusions  of  either  cars  crashing  into  the 
fence  and  then  driving  into  the  airport  or  the 
climbing  of  the  fence  by  pedestrians  who 
could  then  walk  into  the  airport.  Several 
more  fence  sensors  were  also  installed  in  a 
rental  car  area.  A  graphical  user  interface  was 
developed  and  installed  in  the  command 
center  to  aggregate  the  sensors’  output  and 
generate  log  and  time  stamp  for  each  entry. 
An  event-driven  video  recording  capability 
was  added  to  the  smart  fence  functionality  in 
order  to  confirm  the  registered  alarm  with 
the  camera  recordings. 


Figure  9:  Blue  print  of  the  Installed  sensors  at  the 
airport 

The  efficacy  of  the  smart  fence  was 
demonstrated  in  several  scenarios.  In  the 
first,  the  approaching  threat  was  an  incoming 
motorboat,  whose  passengers  disembarked 
on  a  dock  located  at  the  boundary  of  an 
airport.  In  the  second,  the  threat  was  an 
incoming  vehicle  that  drove  through  a  rental 
car  facility  with  the  intention  of  allowing  the 
passengers  to  scale  a  fence  separating  the 
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rental  car  facility  from  the  airport  tower.  In 
the  final  scenario,  the  threats  were  an 
incoming  vehicle  that  drove  off  of  a  major 
thoroughfare,  with  the  intention  of  allowing 
the  passengers  to  scale  a  fence  that  surrounds 
the  end  of  the  airport  runway.  Each  scenario 
provided  different  conditions  and  threat 
levels.  In  the  first,  the  vehicle  engine  sound 
detector  detected  a  motorboat  and  the 
seismic  analyzer  detected  the  intruders' 
footsteps.  In  both  the  second  and  final 
scenarios,  vehicle  engine  sound  detection  was 
common,  but  the  seismic  analyzer  detected 
the  intruders  as  they  approached  the  fence, 
and  the  fence  breaching  sensor  detected 
scaling  attempts.  The  smart  fence 
successfully  identified  the  threats  and 
generated  alarms  before  the  threats 
approached  the  secured  zone. 

To  statistically  analyze  false  positives, 
battery  life,  wireless  issues,  and 
environmental  factors,  the  smart  fence 
technology  at  the  naval  facility  was 
monitored  for  a  period  of  forty-five  days.  For 
the  purpose  of  vehicle  engine  sound 
recognition,  due  to  the  request  of  authorities 
no  test  was  performed  therefore  the  forty-five 
days  test  doesn’t  consider  the  acoustic  sensor. 
The  weekly  average  of  false  positives 
generated  by  other  sensors  has  been 
demonstrated  in  the  Table  4. 


Figure  10:  Daily  report  for  one  of  the  footstep  sensors 
installed  in  front  of  the  boathouse.  Blue  bars  show 
vehicle  recognition  (in  this  care  airplane  either  taxing  or 
taking  off),  red  is  human  footsteps;  x  axis  is  the  time 
and  y  axis  is  the  duration  of  an  event. 


Table  4:  Average  weekly  report  for  the  smart  fence 
sensors 


No.  of 

Awake  Time 

No.  of  False 

Incidents 

(min) 

Positives 

Fence  Sensors 

25 

9 

1.09 

Footstep  Sensors 

1227 

409 

1.89 

Figure  11:  Snapshot  of  a  lawn  mower  colliding  with 
chain-link-fence  at  car  rental  area;  the  sensor  has 
generated  the  letter  “K”  as  an  induction  of  kick. 


CONCLUSIONS  AND 
FUTURE  WORK 

In  this  paper  we  described  development  of 
“smart  fence”  technology  for  intelligent 
acoustic/vibration  security  breach  detection 
and  recognition.  The  “smart  fence”  sensors 
included:  (1)  seismic  event  recognizer  for 
human  footsteps  and  vehicle-caused  ground 
vibrations,  (2)  vibration  sensor  to  detect 
intentional  fence  breaches  and  discriminate 
between  rattle,  kick,  and  climb,  and  (3) 
acoustic  vehicle  sound  recognizer  for 
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detecting  an  approaching  vehicle  and 
identifying  type  of  vehicle. 

The  smart  fence  was  tested  in  controlled 
and  uncontrolled  environments.  In  the 
controlled  environment  the  threat  seismic 
event  recognizer  showed  above  98% 
performance  and  the  average  performance 
was  more  than  98%  and  97%  for  the  fence 
and  vehicle’s  engine  sound  recognizers.  The 
uncontrolled  and  forty-five  days  test  of  the 
sensors  proved  that  all  sensors  have 
extremely  low  false  positives.  Also,  the  test 
coinciding  with  the  local  storm  season 
provided  opportunities  to  measure  the 
performance  of  the  recognizers  in  the 
presence  of  extreme  weather  conditions  such 
as  heavy  rain  and  wind.  Induhitahly,  wind 
and  rain  can  have  significant  impact  on 
sound  and  vibrations.  However,  as  the 
average  weekly  report  summarized  in  the 
Table  4  demonstrates,  the  extreme  weather 
conditions  have  not  generated  a  challenge  to 
the  sensors’  performance. 

During  both  controlled  and  uncontrolled 
tests  we  observed  that  the  seismic  sensors 
perform  close  to  ideal  when  the  threat  is  in 
the  radius  of  sixty  feet.  We  also  found  that  for 
each  panel  of  chain-link  fence  there  is  a  need 
for  one  sensor  regardless  of  width  and  height 
of  the  fence.  The  reliable  detection  range  for 
the  vehicle  engine  sound  recognizer  was  120 
feet  direct  line-of-sight.  These  findings 
suggest  that  the  manufacturing  cost  of  each 
technology  should  be  very  cheap  -  and  easy 
to  install  and  maintain  -  for  the  purpose  of 
protecting  a  several-mile-long  perimeter  of 
an  airport.  All  of  the  designed  technologies 
have  satisfied  the  low  cost  condition. 

Security  breaches  often  follow  certain 
sequences.  As  an  example,  an  intruder  must 
take  a  few  steps  before  disturbing  the  fence.  A 
false  fence  disturbance  detection  by  a  fence 
sensor  due  to  the  wind  can  be  eliminated  if 
the  output  of  the  seismic  sensor  is  combined 
with  the  fence  recognizer.  High-level  data 
fusing  of  the  sensors  of  this  study  not  only 
could  enhance  the  false  positives  and  true 
negatives,  it  could  also  provide  management 
of  the  threats  by  airport  authorities. 
Sequential  modeling  and  analysis  of  detected 
threat  by  smart  fence  sensors  is  the  plan  for 
the  future  work. 
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